Design and Secure Evaluation of Side-Choosing Games

We present an important, general class of new games, called side-choosing games (SCGs), for “gamifying” problem solving in formal sciences using plausibility checking. Applications of SCGs include (1) peer-grading in teaching to (2) studying the evolution of knowledge in formal science communities to (3) organizing algorithm competitions. We view SCGs as a new and general model for formulating formal problems that need to be solved using human computation and our interest in this paper is on how to evaluate a set of SCGs about the same problem fairly and effectively. We observe that a specific kind of collusion, where players lie about their strength and sacrifice themselves, could bias the evaluation of SCGs dramatically. Following the idea of Social Choice Theory in the sense of Arrow, we take an axiomatic approach to guarantee that a specific kind of collusion is impossible. We prove the Collusion-Resistance Theorem and related results as a general principle for designing collusion-resistant evaluations for SCGs. The Collusion-Resistance Theorem is surprising: it tells us to be indifferent to wins but to count certain kinds of losses for scoring players and ranking them. If collusion is not an issue, we offer a family of useful ranking functions which are not collusion-resistant. TO BE DELETED: Limit: 18 pages. July 24-28, ’16 The Netherlands. 11/30/15, Submission deadline: February 23, 2016.EC’16 will be co-located with the 5th World Congress of the Game Theory Society (GAMES 2016), May 30, 2016: Camera-ready version of accepted papers due. Our focus areas: (1) Theory and Foundations and (2) Applied Game Theory. Our topics from CFP: Social Choice and Logic, Collective Intelligence, Game-theoretic models of competitions. Version Feb. 5